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Summary 


Problem  and  Background 

The  military  services  have  high  security  risk  billets  for 
enlisted  accessions  that  must  be  filled  only  by  extremely 
trustworthy  and  reliable  personnel.  To  ensure  that  newly 
assigned  personnel  are  qualified  from  a  security  perspective, 
each  service  uses  various  prescreening  procedures.  in 
addition,  for  candidates  successfully  passing  the  initial 
prescreen,  an  intensive  and  costly  background  investigation  is 
conducted  by  the  Defense  Investigative  Service  and  adjudicated 
by  the  respective  service. 

There  has  been  increasing  DoD  concern  with  espionage 
incidents  and  other  failures  of  security  clearance  policies  to 
protect  classified  information.  There  is  a  need  to  examine 
and  validate  new  procedures  and  approaches,  including 
behavioral  science  tests,  that  could  provide  valid  and 
cost-effective  supplemental  or  alternative  methods  for 
screening  per sonne 1 . 

Objective 

The  objective  of  the  present  study  was  to  examine  the 
validity  of  an  experimental  DoD  prescreening  instrument  in  the 
personnel  security  arena.  More  specifically,  the  Armed 
Services  Applicant  Profile  (ASAP),  a  b  ack  g  ro  u  nd  /  b  i  od  a  t  a 
questionnaire,  was  investigated  to  determine  if  it  could  be 
used  to  predict  the  results  of  background  investigations 
conducted  for  recruits  applying  for  high  security  risk  jobs. 

Approach 

Data  from  an  ongoing  ASAP  validation  study  were  used.  The 
predictor  variables  were  ASAP  item  responses,  along  with  AFQT 
and  years  of  education.  The  subjects  consisted  of  3,257 
enlisted  personnel,  from  an  original  accession  sample  of 
55,675,  who  were  potentially  entering  high  security  risk  jobs. 
These  individuals  had  completed  the  ASAP  as  a  part  of  an 
ongoing  DoD  study.  In  addition,  they  had  successfully  passed 
prescreening  by  their  respective  services  and  had  a  completed 
ba ckground  investigation. 

Classification  of  a  background  investigation  as  an  issue 
case  or  non-issue  case  represented  the  criterion  variable. 
Issue  cases  are  significant  because  they  often  indicate  that 
there  may  be  adverse  information  in  the  person's  background 
that  reflects  on  that  person's  trustworthiness  or  reliability 
and  thus  on  his  or  her  qualifications  to  hold  a  high  security 
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risk  job.  Also,  issue  case  status  has  been  shown  to  predict 
subsequent  unsuitability  discharge  from  the  military  service. 
Approximately  12  percent  of  the  background  investigations  were 
identified  as  issue  cases. 


The  ASAP  predictor  score  was  developed  using  a  weighted 
application  blank  method.  Approximately  one-half  of  the  total 
background  investigation  sample  was  randomly  assigned  to  the 
scoring-key  development  group  while  the  remainder  was  used  as  a 
holdout  group  for  cross-validation  purposes. 

Results 


The  ASAP  scoring  key  correlated  significantly  with  the 
issue  case  criterion  measure  (r  =  .36).  In  addition,  the 

validity  of  the  ASAP  key  held  up  well  across  different 
subgroups  (i.e.,  individual  services,  white/minori ty  , 
male/f emale ) .  Results  indicated  that  almost  24  percent  of  the 
those  individuals  having  the  lowest  20  percent  of  the  ASAP 
scores  had  issue  case  background  investigations.  On  the  other 
hand,  among  the  highest  20  percent  of  ASAP  scorers,  only  4.4 
percent  were  issue  cases.  Overall,  the  ASAP  was  a  far  superior 
predictor  than  either  AFQT  or  years  of  education. 


Analyses  of  the  different  content  categories  of  biodata 
items  on  the  ASAP  indicated  that  issue  case  personnel  were  more 
likely  to  have  used  tobacco,  had  problems  in  school  and  on 
previous  jobs,  and  committed  a  number  of  minor  misbehaviors 
before  entering  the  service.  Finally,  with  regard  to 
adjudication  results  for  Army  and  Air  Force  personnel,  ASAP 
differentiated  between  those  personnel  who  were  granted  a  top 
secret  or  higher  clearance  versus  those  personnel  who  received 
a  secret  clearance  or  were  denied  a  clearance. 

Conclusions 

The  results  supported  the  potential  use  of  ASAP  type 
instruments  as  prescreens  for  personnel  entering  high  security 
risk  jobs.  A  number  of  caveats  were  identified  which  suggest 
that  additional  studies  may  have  to  be  conducted  using  more 
refined  security  criteria.  In  addition,  the  operational 
implications  of  using  prescreening  instruments  need  to  be  more 
clearly  delineated.  Nonetheless,  it  appears  that  behavioral 
science  tests  such  as  the  ASAP  could  potentially  play  an 
important  role  in  the  personnel  security  screening  process. 
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Introduction 


Problem 

The  military  services  have  high  security  risk  billets  for 
enlisted  accessions  that  require  a  top  secret  (TS)  or  sensitive 
compartmented  information  (SCI)  clearance.  These  jobs  are 
usually  found  in  the  communications,  intelligence,  cryptology, 
and  electronic  ratings.  In  addition,  critical  positions  exist 
for  new  enlisted  accessions  in  the  Nuclear  Weapon  Personnel 
Reliability  Programs  run  by  each  of  the  services.  In  order  to 
be  assigned  to  the  above  jobs,  an  intensive  and  costly 
background  investigation  must  first  be  conducted  by  the  Defense 
Investigative  Service.  The  findings  from  this  investigation 
serve  as  the  basis  for  granting  or  denying  the  required 
personnel  security  clearance.  To  reduce  investigative  costs  as 
well  as  to  select  potentially  trustworthy  and  reliable 
personnel,  each  service  initiates  prescreening  procedures  prior 
to  the  initiation  of  a  request  for  a  formal  background 
investigation.  These  procedures  select  out  personnel 
potentially  unqualified  from  a  security  perspective;  the  result 
is  that  background  investigations  are  only  requested  for 
personnel  successfully  passing  the  prescreen. 

There  is  considerable  variability  across  the  services  in 
the  prescreening  procedures  used  for  applicants  and  recruits 
applying  for  high  security  risk  jobs  (Flyer,  1986).  At  the 
present  time,  it  is  not  known  which  service's  approach  is 
working  best;  the  procedures  have  not  been  systematically 
validated  with  regard  to  their  usefulness  and  cost- 
effectiveness.  There  is  a  need  to  conduct  such  validation 
studies  as  well  as  to  examine  new  procedures  and  approaches 
that  could  provide  valid  and  cost-effective  supplemental  or 
alternative  prescreening  methods. 

Objective 

The  objective  of  the  present  study  was  to  examine  the 
validity  of  an  experimental  DoD  prescreening  questionnaire  in 
the  personnel  security  arena.  More  specifically,  the  Armed 
Services  Applicant  Prof ile (ASAP )  was  investigated  to  determine 
if  it  could  be  used  to  predict  the  results  of  background 
investigations  conducted  for  recruits  applying  for  high 
security  risk  jobs.  If  questionnaires  such  as  the  ASAP  are 
found  to  be  valid  predictors  of  personnel  security  criteria, 
they  could  be  used  as  cost-effective  prescreening  devices  to 
supplement  existing  prescreening  approaches  for  high  security 
risk  jobs.  They  could  also  reduce  the  probability  of  assigning 
unreliable  personnel  to  high  security  risk  jobs. 
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Background 


There  has  been  increasing  Department  of  Defense  (DoD) 
concern  with  espionage  incidents  and  other  failures  of  security 
clearance  policies  to  protect  classified  information  as  well 
as  with  the  efficiency  of  the  security  clearance  process.  In 
response,  a  select  panel  (Department  of  Defense,  1982)  was 
formed  to  review  the  DoD  Personnel  Security  Program.  More 
recently,  a  DoD  security  review  commission  (Keeping  the 
Nation's  Secrets,  1985)  was  established  to  review  all  aspects 
of  DoD  security  practices  and  procedures.  While  a  number  of 
new  initiatives  and  recommendations  were  generated,  one  area 
that  received  positive  endorsements  by  both  groups  was 
increased  research  in  the  application  of  behavioral  science 
tests  as  a  part  of  the  prescreening  and  investigatory 
procedures  used  to  approve  security  clearances. 

The  military  services  are  not  systematically  using 
behavioral  science  tests  to  prescreen  recruits  for  high 
security  risk  jobs.  Each  service  has  its  own  prescreening 
procedures,  and  all  make  use  of  some  variant  of  a  prescreening 
interview  in  combination  with  information  from  various 
questionnaires  that  address  previous  drug  use,  involvement  with 
the  law,  financial  problems,  and  other  information  relevant  to 
suitability  for  assignment  to  sensitive  jobs.  Only  the  Air 
Force  uses  psychological  tests  (i.e.,  biographic,  attitude,  and 
personality  items)  as  a  part  of  its  prescreening  process  (see 
Flyer, 1986,  for  a  more  thorough  discussion  of  service 
prescreening  procedures  for  high  security  risk  jobs). 


There  is  a  need  to  assess  whether  behavioral  science  tests 
can  validly  predict  personnel  security  criteria.  If  one  test 
proved  particularly  useful,  it  could  provide  an  effective 
procedure  for  initially  prescreening  personnel  applying  for 
high  risk  security  jobs.  Individuals  successfully  passing  the 
first  prescreen  could  then  be  screened  in  greater  depth  by  the 
current  programs  being  used  by  each  of  the  services.  Actual 
operational  use  of  such  a  test  would  have  to  depend  on  a  number 
of  criteria,  including  cost  benefits  and  the  size  of  the 
applicant  pool.  However,  there  is  first  a  need  to  assess 
whether  such  tests  are  even  predictive  of  important  personnel 
security  criteria. 


Examination  of  the  psychological  test  data  being  gathered 
by  the  Air  Force  would  provide  one  approach  for  initially 
determining  the  validity  of  psychological  tests  in  personnel 
security.  However,  the  focus  of  the  current  effort  was  to  look 
at  an  instrument  that  had  been  administered  to  personnel  in  all 
of  the  services.  This  would  allow  for  a  validity  assessment 
across  the  different  services.  The  Armed  Services  Applicant 
Profile  met  this  requirement,  and  it  is  discussed  in  greater 
detail  below. 
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The  Armed  Services  Applicant  Profile  (ASAP).  The  General 
Accounting  Office  issued  a  1982  report  that  noted  that  all  of 
the  services  were  conducting  research  on  similar  biographical 
(biodata)  questionnaires  for  screening  applicants  to  reduce 
first  term  attrition  (General  Accounting  Office,  1982).  In 
response  to  this  report  and  to  Congressional  interest  in 
enlisted  screening  procedures  that  place  less  emphasis  on  high 
school  graduate  status,  DoD  initiated  a  study  to  develop  and 
validate  a  biodata  questionnaire  suitable  for  use  by  all 
services  as  an  adaptability  screening  instrument  (Atwater, 
Walker  &  Weaver,  1987).  The  resultant  biodata  questionnaire 
was  the  Armed  Services  Applicant  Profile  (ASAP).  An  ASAP 
validation  study  against  a  criterion  of  first  year  attrition  is 
currently  being  conducted  by  the  Navy  Personnel  Research  and 
Development  Center  in  San  Diego,  California  (Trent,  in 
preparation ) . 


The  ASAP  could  provide  an  ideal  initial  vehicle  for 
examining  the  predictive  validity  of  one  type  of  behavioral 
science  test,  the  biographical  questionnaire,  against 
personnel  security  criteria.  Although  the  test  was  not 
specifically  designed  as  a  personnel  security  prescreening 
instrument,  the  ASAP  includes  a  wide 
items.  Personal  history  items  have 
predict  a  variety  of  complex  behaviors 
(Ghiselli,  1966;  Asher,  1972;  Owens, 

Also,  the  fact  that  the  ASAP  was 
adaptability  to  military  life  makes  it  potentially  relevant  to 
personnel  security.  Personnel  reliability  and  suitability  are 
two  criteria  important  to  personnel  security  prescreening. 
Clearly,  failure  to  complete  the  first  year  of  military  service 
due  to  adaptability  problems  indicates  potential  unreliability 
and  unsuitability  for  security  responsibilities  in  sensitive 
jobs.  Specific  personnel  security  criteria  encompass  a  number 
of  areas  and  these  are  discussed  in  greater  detail  below. 


range  of  biographical 
been  demonstrated  to 
in  the  private  sector 
1976;  Cascio,  1982). 
designed  to  predict 


Personnel  security  criteria.  The  basic  goal  of  personnel 
security  policy  is  to  ensure  that  no  person  is  given  access  to 
classified  information  or  assigned  to  sensitive  duties  unless  a 
favorable  determination  has  first  been  made  regarding  his  or 
her  loyalty,  trustworthiness,  reliability,  and  judgment.  For 
military  enlisted  accessions  entering  high  security  risk  jobs, 
there  is  an  initial  cognitive  screening  to  qualify  personnel 
for  training  schools.  In  addition,  service  prescreening 
programs  and  the  background  investigation  are  used  to  achieve 
the  above  personnel  security  objective. 

During  both  the  prescreening  and  background  investigation 
phase,  an  attempt  is  made  to  identify  adverse  background 
information  relevant  to  personnel  security.  This  usually 
includes  information  on  financial  irresponsibility,  criminal 
conduct,  sexual  misconduct,  mental  or  emotional  illness, 
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hostile  foreign  country  connections,  subversive  activity, 
alcohol  and  drug  abuse,  and  previous  security  violations 
(Department  of  Defense,  1979).  The  final  granting  of  a 
security  clearance  affirms  that  an  individual  has  no 
significant  adverse  background  information  that  would  make  him 
or  her  ineligible  for  access  to  classified  information. 

Clearly,  the  key  personnel  security  criterion  is  whether 
or  not  an  individual  demonstrates  reliability,  trustworthiness, 
good  judgment,  and  loyalty  in  the  actual  handling  and  use  of 
classified  information.  Failure  to  do  so  could  be  manifested 
at  one  level  in  excessive  security  violations  and  at  the 
extreme  in  the  deliberate  compromise  of  classified  information, 
including  espionage.  From  a  test  validation  perspective, 
neither  of  these  criteria  are  useful.  Espionage  has  such  a  low 
base  rate  that  no  existing  behavioral  science  test  could 
demonstrate  significant  prediction.  With  regard  to  security 
violations,  the  base  rate  is  somewhat  higher  but  still  very  low 
for  reliable  prediction;  also,  historical  security  violation 
data  are  not  systematically  maintained  on  centralized  data 
bases  and  therefore  cannot  be  accessed  for  large  scale 
validation  studies . 

For  military  enlisted  personnel,  a  potential  surrogate 
measure  could  be  early  discharge  from  high  security  risk  jobs 
for  unsuitability  reasons.  Flyer  (1986,  pp .  12-13)  has 
elaborated  on  this  criterion: 

Since  a  major  objective  of  the  Defense  personnel 
security  program  is  to  identify  people  who  are 
suitable  for  assignment  to  highly  sensitive 
positions,  later  unsuitability  discharge  among  those 
selected  for  these  positions  could  be  viewed  in  part 
as  failure  of  the  screening  process.  Also,  and 
critically  important,  personnel  discharged  from 
highly  sensitive  positions  for  unsuitability  pose  a 
special  security  problem.  A  number  of  those 
discharged  are  likely  to  be  quite  bitter  as  a  result 
of  their  experience  during  military  service,  many 
would  be  knowledgeable  of  sensitive  equipment  and 
procedures,  and  almost  all  would  be  facing  some 
degree  of  financial  uncertainty  on  their  return  to 
civilian  life. 

For  the  current  study,  the  problem  with  using 
unsuitability  discharges  as  the  criterion  is  that  reliable 
measurement  necessitates  that  individuals  be  tracked  for  one  to 
four  years  after  their  entry  into  active  service.  While  such  a 
longitudinal  study  could  be  undertaken  in  the  future,  there  was 
a  need  to  identify  a  more  proximal  criterion.  The  background 
investigation  provided  such  a  measure. 
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cases . 


Background  investigation  issue  cases.  An  Entrance 
National  Agency  Check  ( ENT  NAC )  Ts  required  for  all  non-prior 
service  enlisted  accessions.  In  addition,  personnel  entering 
high  security  risk  jobs  must  have  the  in-depth  background 
investigation  (BI)  completed.  Whereas  the  ENTNAC  consists  of  a 
search  of  records  held  by  various  federal  agencies,  the  BI 
includes  checks  of  local  police  records,  a  credit  check, 
educational  and  employment  checks,  interview  of  references,  and 
in  some  cases,  a  subject  interview.  The  Manual  for  Personnel 
Security  Investigations  (Defense  Investigative  Service,  1985) 
outlines  the  basic  requirements  for  conducting  the  different 
types  of  Bis.  In  those  cases  where  potential  derogatory 
information  is  uncovered  during  the  normal  investigative 
procedures,  an  expanded  investigation  is  often  conducted  in  the 
problem  area.  This  situation  is  called  an  issue  case.  It 
should  be  noted  that  the  investigatory  process  does  not  pass 
judgment  on  the  individual;  rather,  it  merely  uncovers  as  much 
relevant  information  as  possible,  given  certain  resource 
constraints  .  Actual  review  of  the  information  for  granting 
clearances  occurs  during  a  later  adjudicative  phase. 


Issue  cases  are  important  because  they  signify  that  there 
may  be  adverse  information  in  the  person's  background  that 
reflects  on  that  person's  trustworthiness  or  reliability  and 
thus  on  his  or  her  qualifications  to  hold  a  high  level  security 
clearance.  Identification  of  an  issue  case  is  not  a  perfect 
measure  of  whether  or  not  an  investigation  yielded  significant 
ad vers e  information : 


1.  When  an  issue  case  investigation  is  expanded,  it 
sometimes  turns  out  (perhaps  10  percent  of  the  time)  that  the 
case  is  favorably  resolved.  That  is,  the  original  assessment 
that  there  was  adverse  information  did  not  hold  up  when  the 
specific  incidents  were  investigated  in  depth.  Nonetheless, 
the  case  still  remains  an  issue  case,  since  it  did  require  an 
expanded  investigation . 


2.  An  initial  investigation  may  yield  adverse 
information.  Many  investigations  may  not  become  issue  cases 
even  though  they  contain  valid  derogatory  information. 


3.  It  is  possible  that  there  may  be  resistance  to  the  use 
of  the  issue  case  label.  The  focus  of  the  investigative 
process  is  to  provide  relevant  information  on  different  aspects 
of  a  person's  background  but  not  to  make  a  judgment  as  to  the 
severity  of  that  information  for  security  clearance  decisions. 
The  use  of  the  issue  case  label  may  connote  such  a  judgment  and 
thus  may  not  be  used  in  some  instances. 

4.  Finally,  even  when  an  issue  case  contains  valid 
adverse  information,  it  may  still  be  positively  adjudicated 
during  a  later  review  process.  The  adjudicator  may  determine 
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that  the  information  is  not  significant  enough  to  deny  a 
security  clearance. 

The  above  limitations  might  argue  for  focusing  only  on 
those  small  proportion  of  issue  cases  that  are  negatively 
adjudicated.  In  these  instances,  there  would  be  no  doubt  that 
the  case  contained  adverse  information.  However,  despite  the 
above  problems,  issue  case  status  still  appears  to  be  an 
important  criterion  independent  of  the  adjudication  results. 
Flyer  (1985)  conducted  analyses  that  pointed  out  the  importance 
of  issue  case  status  for  military  enlisted  accessions.  He 
analyzed  data  for  over  1.5  million  non-prior  service  male 
enlisted  personnel  entering  the  military  services  during  the 
period  FY-74  to  FY-78.  About  150/000  or  ten  percent  of  this 
population  had  background  investigations  completed  during  their 
initial  tour  of  duty.  For  this  subsample,  he  determined 
unsuitability  discharge  rates  for  issue  cases  and  non-issue 
cases  as  a  function  of  high  school  graduation  status.  These 
results  are  reproduced  in  Table  1. 

Enlistees  whose  background  investigations  resulted  in 
issue  cases  were  much  more  likely  to  be  separated  or  discharged 
early  for  unsuitability  reasons.  This  relationship  held  up 
across  all  services  and  for  both  high  school  and  non-high 
school  graduates.  Flyer’s  results  strongly  indicate  that  issue 
case  status  is  an  important  predictor  of  future  unsuitability 
discharges.  They  also  argue  for  using  issue  case  status  of 
background  investigations  as  one  of  the  criteria  for  the 
initial  validation  of  potential  prescreening  tests. 

TABLE  1 


Unsuitability  Discharge  Rates  by  Background  Investigation  Results3 


Group 

] 

Army 

Percent  Attrition 
Navy 

Rate 

Air  Force 

High  School  Graduates 

Issue  Cases 

36.2 

31.8 

61 . 1 

Non-Issue  Cases 

17.7 

22.6 

26.5 

Non-High  School  Graduates 

Issue  Cases 

54.9 

58.8 

61 . 1 

Non-Issue  Cases 

34.7 

43.7 

47.2 

aData  represent  all  male  non-prior  service  enlisted  personnel  entering 
the  military  services  during  the  period  FY-74  to  FY-78  who  had  background 
investigations.  Data  taken  from  Flyer,  1985.  (N  =  150,000) 
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Approach 


The  approach  used  in  this  study  was  to  use  existing  data 
from  an  ongoing  ASAP  validation  study  to  examine  the  usefulness 
of  biodata  items  as  predictors  of  a  personnel  security 
screening  criterion.  The  prediction  variables  were  ASAP  item 
responses,  along  with  AFQT  and  education,  while  the  criterion 
variable  represented  the  results  of  a  background  investigation 
conducted  for  each  enlisted  accession  classified  to  enter  a 
sensitive  billet. 

Ins t rumen t/Predi ctors 

Biodata  instrument  -  ASAP.  The  ASAP  has  two  parallel 
forms  with  each  form  consisting  of  130  biodata  items  developed 
from  previous  research  (Atwater  et  al . ,  1987).  Each  form 
contains  60  common  items  in  Part  1  that  duplicate  the  Military 
Applicant  Profile  (MAP),  a  biodata  instrument  used  to  screen 
non-high  school  graduate  applicants  to  the  U.  S.  Army  (Frank  & 
Erwin,  1978).  Part  2  of  each  form  contains  an  additional  30 
common  items  plus  40  unique  items.  The  majority  of  Part  2 
items  are  from  the  Recruit  Background  Questionnaire  developed 
at  the  Navy  Personnel  Research  and  Development  Center  (Atwater 
&  Abrahams,  1983  ).  The  130  items  fall  into  a  number  of 
rational  content  clusters,  and  this  distribution  for  ASAP  Form 
1  items  is  shown  in  Table  2. 


TABLE  2 


Content  Clusters  for  Items 
on  the  Armed  Services  Applicant  Profile  (Form  1) 


Content  Area 

Number 
of  Items 

Content  Area 

Number 
of  Items 

Self  Description 

Childhood /Family 
Experiences 

8 

School  Background 

18 

Self -Esteem 

8 

Work  History 

16 

Athletic  Activity 

5 

Minor  Misbehaviors 

1 1 

General  Perceptions 

ij 

Social  Activities 

9 

Alcohol/Tobacco  Use 

3 

Enlistment  Influences 

9 

Hobbies /Interests 

3 

Service  Perceptions 

9 

Family’s  Socio- 

Economic  Status 

3 

As  can  be  seen  in  Table  2,  ASAP  items  cover  a  wide  range 
of  background  areas,  with  the  greatest  concentration  in  the 
general  self-description,  school  background,  and  work  history 
clusters.  All  potential  ASAP  items  were  reviewed  extensively 
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to  eliminate  items  likely  to  be  perceived  as  offensive  by 
applicants  or  biased  against  subgroups.  In  addition,  highly 
intrusive  items  were  also  excluded  from  the  final 
questionnaire.  Response  options  to  items  vary  as  a  function  of 
the  type  of  question  but  always  include  from  three  to  five 
fixed  response  choices;  over  70  percent  of  the  items  offer  five 
response  choices.  Some  item  response  options  represent 
different  qualitative  choices  while  others  approximate 
Likert-type  scales.  It  should  be  noted  that  the  three 
socio-economic  status  items,  two  of  the  school  background 
items,  and  two  of  the  self-description  items  are  currently 
being  excluded  from  scoring  in  the  DoD  ASAP  study. 


Because  the  ASAP  is  currently  undergoing  experimental 
validation  and  because  of  its  potential  operational  use  by  DoD 
in  the  future,  specific  ASAP  items  are  not  presented  in  this 
report.  However,  the  following  item  is  representative  of  the 
kind  of  self-report  questions  that  constitute  the  ASAP: 

What  were  your  grades  in  high  school? 


A. 

B. 

C. 


D. 

E. 


Mostly  A 1 s 
Mostly  A' s 
Mostly  B 1 s 
Mostly  C 1 s 
Mostly  D 1 s 


and  B ' s 
and  C 1 s 
and  D 1 s 
and  below 


Items  range  from  those  that  could  be  potentially  verified  (such 
as  the  foregoing  sample  item)  to  more  subjective  and 
non-verif iable  items  such  as  "How  much  do  you  smoke?",  or  "How 
do  you  feel  when  people  tell  you  what  to  do?". 


Other  predi ctors .  In  order  to  determine  whether  the  ASAP 
was  a  better  predictor  of  issue  cases  than  more  traditional 
screening  variables,  two  additional  measures  were  included  in 
the  study.  These  included  the  Armed  Forces  Qualification  Test 
(AFQT)  score,  a  general  aptitude  measure  used  to  screen  all 
military  applicants,  and  a  three  tier  measure  of  education 
level  (high  school  graduate,  alternative  certificate  holder, 
and  non-high  school  graduate  (see  Eitelberg,  Laurence,  Waters, 
&  Perelman,  1984).  The  above  data  were  obtained  for  all 
accessions  who  completed  the  ASAP  by  matching  those  records 
with  service  master  tapes  maintained  by  the  Defense  Manpower 
Data  Center  ( DMDC )  in  Monterey,  California. 

ASAP  Admin i strati on 


All  military  applicants  in  the  continental  United  States 
for  the  3-month  period  from  December  1984  to  February  1985 
completed  the  ASAP  at  either  a  Mobile  Enlistment  Testing  Site 
or  a  Military  Enlistment  Processing  Station  as  a  part  of  the 
DoD  validation  study.  A  total  of  205,968  completed  the  ASAP, 
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which  required  approximately  60  minutes  to  administer.  A 
comparison  of  demographic  data  for  this  3-month  sample  with  all 
FY  1985  applicants  yielded  no  substantial  differences  with 
regard  to  race  and  gender,  while  some  ov  er- samp  1  i  ng  of  high 
school  graduates  and  low  AFQT  scorers  was  evident  (Atwater  et 
al . ,  in  press ) . 

It  is  important  to  note  that  military  applicants  rather 
than  new  enlistees  were  tested  in  order  to  gain  realistic 
self-responding  data  that  included  distortion  to  "look  good" 
(Walker,  1985).  Thus,  empirically  derived  scoring  keys  based 
on  the  applicant  data  could  control  for  such  distortion. 

Subjects 

ASAP  sample  .  The  ongoing  DoD  ASAP  study  is  tracking  the 
first  year  attrition  of  all  non-prior  service,  active  duty 
accessions.  For  the  current  study,  a  cutoff  date  of  30  June 
1986  yielded  a  non-prior  service  accession  sample  of  55,675 
enlisted  personnel.  This  initial  sample  excluded  3,490  prior 
service  accessions  and  73,327  non-accession  applicants  from  a 
total  active  duty  applicant  sample  of  132,492. 

As  mentioned  earlier,  there  are  a  number  of  jobs  for 
incoming  enlisted  accessions  that  require  a  TS  or  SCI 
clearance.  In  addition,  critical  positions  for  incoming 
enlisted  personnel  also  exist  as  part  of  the  Nuclear  Weapon 
Personnel  Reliability  Program.  For  the  purpose  of  the  present 
report,  the  above  billets  have  been  labeled  as  high  security 
risk  jobs.  They  represent  job  categories  for  which  the 
individual  services  conduct  special  prescreening  and  that 
require  background  investigations  be  completed  (and  positively 
adjudicated)  for  personnel  prior  to  initial  assignment. 

Background  investigation  sample.  The  current  focus  was  on 
enlisted  personnel  applying  for  these  high  security  risk  jobs 
that  required  a  DIS  background  investigation.  Since  this  study 
was  initiated  after  the  recruits  had  successfully  passed  the 
prescreening  procedures  used  by  each  service,  there  was  no  way 
to  retroactively  identify  those  recruits  who  were  prescreened 
out.  Selective  data  on  all  background  investigations 
(including  issue  case  status)  is  maintained  on  the  Defense 
Central  Index  of  Investigations  (DCII).  This  data  base  is 
warehoused  at  DMDC .  In  order  to  identify  ASAP  accessions  with 
background  investigations,  DMDC  merged  existing  ASAP  files  from 
the  ongoing  DoD  study  with  the  DCII  files.  This  merging 
yielded  3,275  enlisted  personnel  who  had  both  ASAP  data  and  a 
completed  background  investigation.  Thus,  the  subjects 
selected  for  this  study  were  the  prescreened  subsample  (N 
=  3,257)  of  the  55,765  ASAP  non-prior  service  accessions  for 
whom  a  background  investigation  was  both  requested  by  the 
individual  services  and  completed  by  the  Defense  Investigative 
Service.  A  breakout  of  this  sample  by  the  different  services 
is  presented  in  Table  3. 
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TABLE  3 


Background  Investigation  Sample  by  Service 


Service 

Total 

Accessions 

N 

Accessions  with 

Background  Investigations 

N  * 

Army 

25,136 

882 

3-5 

Navy 

11,963 

1,011 

8.5 

Air  Force 

12,350 

1,259 

10.2 

Marines 

6,226 

1.05 

1 .7 

Total 

55,675 

3,257 

5.9 

Approximately  six  percent  of  the  accessions  had  a  formal 
background  investigation.  The  Air  Force  had  the  largest 
proportion  (10.2%)  of  accessions  with  investigations  while  the 
Marines  had  the  lowest  percentage  (1.7%).  A  comparison  was 
made  between  the  demographic  characteristics  of  the  total 
accession  sample  and  the  subsample  for  whom  a  background 
investigation  was  completed.  These  data  are  shown  in  Table  4. 

TABLE  4 

Comparison  of  Total  Accession  Sample  with  Background  Investigation 
Subsample  on  Selected  Demographic  Variables 


Percent  of  Total 


Subgroup 
(N  =  55,675) 

All  Accessions 
(N  =  3,257) 

Accessions  with 

Background  Investigations 

High  School  Status 

No  Certificate 

5.11 

1 .8 

Alternative  Credential  6.2 

2.0 

High  School  Diploma 

88.11 

96.2 

Gender 

Female 

111.0 

22.0 

Male 

86.0 

78.0 

Race 

Black 

20.1 

16.7 

V.Thite 

75.0 

79-5 

Other 

4.9 

3-8 

AFQT 

I  (93-99) 

3-5 

8.8 

II  (65-92) 

31 .6 

45.5 

Ilia  ( 50—6-4 ) 

24.2 

20.8 

Illb  (31-119) 

32.6 

22.4 

IV  (21-30) 

8.1 

2.4 
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The  background  investigation  subsample  differed  from  the 
total  accession  sample  with  fewer  minorities  (20.5%  vs  25.0%), 
more  females  (22.0%  vs  14.0%),  more  high  school  graduates 
(96.2%  vs  88.4%),  and  higher  AFQT  scores.  The  higher  education 
and  AFQT  composition  of  the  background  investigation  subsample 
reflected  initial  prescreening  for  training  schools  (i.e.,  jobs 
that  require  a  background  investigation  also  require  a 
post-boot  camp  training  school  with  aptitude  cut-off  scores). 

Criterion;  Background  Investigation  Issue  Cases 

Classification  of  a  background  investigation  as  an  issue 
case  or  non-issue  case  represented  the  criterion  variable. 
Table  5  presents  a  breakout  of  the  percentage  of  issues  cases 
for  the  background  investigation  subsample  (N_  =  3,257)  by 
service  and  by  selected  demographic  variables. 

TABLE  5 


Percentage  of  Issue  Cases 
by  Service  and  Selected  Demographic  Variables 


Subgroup 

Number  of 
Investigations 

Percentage  of 

Issue  Cases 

Service 

Array 

882 

12.0 

Navy 

1,011 

17.8 

Air  Force 

1,259 

8.0 

Marines 

105 

7.6 

Gender 

Female 

715 

9-8 

Male 

2, 542 

12.8 

Race 

Black 

545 

14.3 

White 

2,587 

11.7 

Other 

125 

12.0 

High  School  Status 

No  Certificate 

60 

31 .6 

Alternative  Credential 

65 

21.5 

High  School  Diploma 

3,132 

1 1.6 

AFQT 

I  (93-99) 

287 

1 1 .5 

II  (65-92) 

1,482 

10.1 

Illb  (50-64) 

679 

13-4 

Ilia  (31-49) 

731 

14.5 

IV  (21-30) 

78 

17-9 

Total 

3,257 

12.1 

Approximately  12  percent  of  the  background  investigations 
were  identified  as  issue  cases,  with  this  rate  varying  from  a 
high  of  17.8  percent  for  the  Navy  to  a  low  of  7.6  percent  for 
the  Marines.  In  general,  the  issue  case  rates  were  very 
similar  across  the  different  demographic  variables.  However, 
there  were  large  differences  between  the  percentage  of  issue 
cases  for  non-high  school  graduates  (31.6%),  alternative 
certificate  holders  (21.5%),  and  high  school  graduates  (11.6%). 
It  should  be  noted  that  over  96  percent  of  the  total  sample  had 
a  high  school  diploma. 

Ad judi cation .  As  mentioned  earlier,  the  findings  from  the 
background  investigations  are  adjudicated  separately  by  each  of 
the  services  prior  to  granting  a  security  clearance.  The 
results  of  this  adjudication  process  in  terms  of  either 
granting  or  denying  a  clearance  are  coded  on  the  DCII  files  for 
Army  and  Air  Force  personnel.  Clearance  data  for  Navy 
personnel  has  just  recently  been  added  to  the  DCII  files  but 
does  not  include  historical  data  relevant  to  the  Navy  ASAP 
sample . 

Although  the  primary  focus  of  the  current  study  was  on  the 
issue  case/non-issue  case  criterion,  it  was  of  interest  to 
determine  whether  the  ASAP  would  also  predict  adjudication 
results.  Thus,  for  Army  and  Air  Force  personnel  only,  DCII 
files  were  matched  with  the  ASAP  files  to  obtain  data  relevant 
to  final  adjudication/clearance  decisions. 

Scoring  Key  Development 

Given  the  small  sample  size  in  the  low  criterion  group 
(i.e.,  those  personnel  with  issue  case  background 
investigations),  and  in  order  to  double  the  N,  the  90  common 
items  on  both  Form  1  and  Form  2  were  treated  as  a  single 
shortened  questionnaire.  Consistent  with  the  ongoing  DoD 
study,  six  of  these  common  items  were  discarded  because  of 
possible  bias  against  subgroups.  The  result  was  that  the  3,257 
accessions  who  had  background  investigations  were  now  treated 
as  one  sample  who  had  completed  a  shortened  version  of  the 
ASAP.  All  futures  references  to  the  ASAP  in  this  report  refer 
to  this  common  core  of  84  items. 

Approximately  52  percent  (N  =  1,679)  of  the  background 
investigation  accessions  were  randomly  assigned  to  a 
scoring-key  development  sample;  the  remainder  (N  =  1,578)  were 
assigned  to  a  holdout  sample  for  statistical  cross-validation 
purposes.  The  ASAP  predictor  score  was  developed  using  the 
"weighted  application  blank"  method  (England,  1971).  The  Navy 
Personnel  Research  and  Development  Center's  KEYCON  program 
(Abrahams,  Neumann,  &  Rimland,  1973)  was  used  to  select  items 
for  the  scoring  key.  This  was  accomplished  within  the  key 
development  sample  by  comparing  for  each  item  choice  the 
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percentage  of  high  criterion  respondents  (relative  to  the  total 
high  criterion  group)  with  the  percentage  of  low  criterion 
respondents  (relative  to  the  total  low  criterion  group).  This 
percent  difference  was  then  used  as  the  weighted  score  for  that 
respons  e . 

Three  different  keys  were  developed,  based  on  response 
differences  that  equaled  or  exceeded  one,  five,  or  ten  percent. 
Using  the  five  percent  difference  key  as  an  example,  if  60.3 
percent  of  the  high  criterion  group  in  the  key  construction 
sample  endorsed  a  particular  response  option  while  only  52.1 
percent  of  the  low  criterion  group  endorsed  it,  that  option 
received  a  weight  of  +8.2.  Similarly,  if  30.3  percent  of  the 
high  criterion  group  endorsed  another  response  option  while 
42.8  percent  of  the  low  criterion  group  endorsed  it,  that 
option  received  a  weight  of  -12.5.  For  this  key,  percent 
differences  less  than  five  percent  were  scored  as  zero.  An 
individual's  total  ASAP  score  consisted  of  the  sum  of  the 
response  weights  for  those  item  responses  endorsed. 
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Results  and  Discussion 


ASAP  Validities 

Table  6  presents  correlations  (between  ASAP  scores  and  the 
issue  case  criterion)  from  the  key  development  and  cross- 
validation  samples  for  the  three  different  scoring  keys  based 
on  one  percent  (Key  A)  ,  five  percent  (Key  B),  and  ten  percent 
(Key  C)  item  response  differences.  These  correlations  were  not 
corrected  for  indirect  restriction  of  range  of  the  ASAP  scores 
resulting  from  the  service  security  prescreening  procedures; 
accurate  estimates  of  the  amount  of  restriction  were  not 
available.  All  point  biserial  correlations  were  corrected  to 
biserial  correlations  because  of  the  extreme  split  (90/10)  on 
the  dichotomous  criterion. 


TABLE  6 

ASAP  Validities  for  Different  Scoring  Keys 


Scoring  Items  Item  Key  Development  Cross-Validation 


Key 

%  D 

Scored 

Responses 

Sample  (N 
r 

_  =  1679) 

Sample  (N 
r 

=  1578) 

pbis 

bis 

^bis 

bis 

A  (1*) 

83 

250 

.27 

.nn 

.22 

•  36 

B  (50 

ill 

62 

.21 1 

•  39 

.22 

•  36 

C  (10%) 

8 

10 

.18 

.29 

.21 

•  3H 

Note:  All 

point 

biserial  and 

biserial 

correlations  are 

significant  (p< .001 ) . 


The  cross  validation  biserial  correlations  were  all 
significant  (jo  <  .001)  and  were  identical  for  Key  A  and  Key  B 
(r  =  .36)  and  slightly  lower  for  Key  C  (r  =  .34).  Since  Key  B, 
based  on  the  five  percent  differences,  yielded  an  identical 
cross  validity  as  the  one  percent  key,  yet  required  only  41 
scored  items  (as  opposed  to  83  items  for  Key  A),  it  was  used  to 
further  examine  the  relationship  between  the  ASAP  and  issue 
case  rates.  Figure  1  graphically  illustrates  this  relationship 
by  quintiles  for  the  total  cross-validation  sample. 
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Figure  1.  Issue  case  rate  by  ASAP  score  quintiles  for  cross 
validation  group  (N=l,578). 

Figure  1  shows  that  almost  24  percent  of  those  individuals 
having  the  lowest  20  percent  of  the  ASAP  scores  had  issue  case 
background  investigations.  On  the  other  hand,  among  the 
highest  20  percent  of  ASAP  scorers,  only  4.4  percent  had  issue 
case  background  investigations.  Overall,  the  higher  the  ASAP 
quintile,  the  lower  the  rate  of  issue  cases.  Interestingly, 
approximately  42  percent  of  the  issue  case  personnel  scored  in 
the  lowest  20  percent  quintile  on  the  ASAP.  In  other  words,  if 
we  eliminated  the  lowest  20  percent  of  the  ASAP  scorers,  about 
42  percent  of  the  issue  cases  would  be  eliminated. 

ASAP  validities  were  also  examined  for  subgroups  of  the 
cross-validation  sample.  These  data  are  presented  in  Table  7 
using  the  five  percent  difference  scoring  key. 

Within  the  different  services,  the  ASAP  validities  were 
significant  (£  <  .001)  for  the  Army,  Navy,  and  Air  Force 
samples  and  were  significantly  higher  (£  <  .01)  for  Army  (r  = 
.40)  and  Navy  (r  =  .38)  personnel  than  for  Air  Force  personnel 
(r  =  .22).  Because  of  the  small  sample  size,  no  significant 
results  were  obtained  for  the  Marine  sample.  Significant  and 
similar  validities  (r  =  .35  or  .36,  £  <  .001)were  obtained  for 
females,  males,  whites,  and  non-whites.  Overall,  the  validity 
of  the  ASAP  key  held  up  well  across  different  subgroups. 
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ASAP  Validities  by  Subgroups 
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Overall  182  1,396  11.5  37.0  75.3  54.5  56.4  .36  .001 


Content  Categories 


Table  8  rank  orders  different  content  categories  of 
biodata  items  by  the  degree  to  which  they  discriminated  between 
issue  case  and  non-issue  case  personnel.  The  mean  item 
discrimination  was  computed  for  13  rational  content  areas 
representing  the  84  ASAP  items.  This  measure,  for  a  given 
content  cluster,  consisted  of  summing  the  absolute  values  of 
the  percentage  differences  in  item  choice  responding  between 
the  issue  case  and  non-issue  case  personnel  and  then  dividing 
by  the  number  of  items  in  that  cluster. 

TABLE  8 


Rank  Order  of  Content  Categories 
by  Mean  Item  Discrimination 


Content  Content 

Rank  Cluster 

Number  of 

Items 

Mean  Item 
Discrimination3 

i 

Tobacco  Use 

2 

60.95 

2 

Minor  Misbehaviors 

8 

30.2^1 

3 

Work  History 

9 

22.00 

H 

School  Background 

12 

19.18 

5 

Service  Perceptions 

7 

16.57 

6 

Social  Activities 

9 

15.116 

7 

Enlistment  Influences 

6 

13-27 

8 

Athletic  Activity 

5 

12.90 

9 

Hobbies /Interests 

3 

12.23 

10 

Sel f-Descriptions 

13 

12.01 

1 1 

Chi Id hood /Family  Experiences 

6 

11.95 

12 

General  Perceptions 

1 

11.90 

13 

Sel f-Esteem 

3 

7.23 

— 

Total  Scored  Items 

8H 

18.91 

a  This  measure  represents  the  absolute  value  of  the  percentage 
differences  between  the  issue  case  and  non-issue  case  groups  summed  across 
response  choices  in  a  content  cluster  and  then  divided  by  the  number  of 
items  in  that  cluster. 

By  far,  the  content  cluster  that  showed  the  greatest 
discrimination  addressed  tobacco  use  (the  alcohol  use  item  that 
was  originally  part  of  this  cluster  was  not  one  of  the  84  ASAP 
items  common  across  the  two  forms).  The  second  best 
discriminator  was  minor  misbehaviors,  followed  by  work  history 
and  school  background.  The  other  nine  categories  ranged  from  a 
high  of  16.57  for  items  on  service  perceptions  to  a  low  of  7.23 
for  self-esteem  items. 

In  order  to  provide  more  concrete  examples  of  the  types  of 
items  that  discriminated  between  the  issue  case  and  non-issue 
case  personnel,  the  same  analytical  technique  was  used  to  rank 
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order  the  15  items  showing  the  greatest  between-group 
discrimination.  These  results  are  shown  in  Table  9. 

TABLE  9 


ASAP  Items  Showing  Greatest  Between-Group  Discrimination 


Ran  k 

Issue  Case  Personnel 

More  Likely  to: 

Content 

Cl  uster 

Total  Item 
Discrimination3 

1 

Smoke 

12 

61.60 

2 

Start  smoking  at  younger  age 

12 

60.30 

3 

Be  suspended  from  school  at 
younger  age 

14 

143.8O 

Have  a  tattoo 

14 

143.6O 

5 

Be  fired  from  a  civilian  job 

3 

143.6O 

6 

Be  expelled  from  school 

14 

142.60 

7 

Have  thought  about  quitting 
school 

2 

30.80 

8 

Have  parents  who  were  unhappy 
about  their  school  grades 

2 

30.140 

9 

Have  lower  grades  in  school 

2 

29-^0 

10 

Stay  out  more  on  weekends 

5 

27-140 

11 

Have  been  in  trouble  with 
police 

14 

26.00 

12 

Have  had  more  full  time  jobs 

3 

25.30 

13 

Not  plan  on  making  service  a 
career 

7 

25-30 

111 

Have  school  grades  lower  than 
they  expected 

2 

214.00 

15 

Have  applied  for  more  jobs 

3 

23.30 

a 

This  measure  represents  the 

absolute  value 

of  the  percentage 

difference  between  the  issue  case  and  non-issue  case  groups  summed  across 
the  response  choices  for  an  item. 

Those  personnel  whose  background  investigations  resulted 
in  issue  cases  were  more  likely  to  smoke  and  to  have  started 
smoking  at  a  younger  age.  They  were  more  likely  (when  compared 
to  non-issue  case  personnel)  to  have  (1)  made  lower  grades  than 
either  they  or  their  parents  expected,  (2)  made  lower  grades 
than  non-issue  case  personnel,  (3)  experienced  problems  in 
school,  including  having  been  expelled  and/or  suspended  and 
having  thought  about  quitting  school,  (4)  held  and/or  applied 
for  more  full  time  jobs,  (5)  been  fired  from  a  civilian  job, 
(6)  gotten  a  tattoo,  (7)  stayed  out  more  on  weekends,  and  (8) 
decided  not  to  make  the  service  a  career.  This  profile 
suggests  that  the  typical  individual  with  an  issue  case 
background  investigation  had  turned  away  from  an  unhappy  school 
situation  to  seek  an  outside  job  and  socio-economic  rewards. 
The  fact  that  the  individual  had  problems  both  in  school  (e.g., 
more  likely  to  have  been  expelled  or  suspended)  and  on  the  job 
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(e.g.,  more  likely  to  have  been  fired)  suggests  a  general 
factor  of  unreliability.  Overall,  this  profile  does  not 
represent  the  ideal  type  of  person  that  the  military  services 
would  want  to  assign  to  a  high  security  risk  job. 
Surprisingly,  all  of  these  individuals  had  successfully  passed 
the  service  prescreening  procedures  since  they  all  had  a 
background  investigation . 

Other  Predictors 


In  order  to  assess  whether  the  ASAP  was  a  more  valid 
predictor  of  issue  cases  than  other  commonly  used  measures, 
AFQT  scores  and  years  of  education  were  examined.  Table  10 
presents  the  results  of  the  analyses  assessing  the  relationship 
for  personnel  in  the  cross-validation  sample  between  these 
predictors  and  the  issue  case  criterion. 

TABLE  10 


Relationship  Between  Predictors  and  Issue  Case  Criterion 
( Cross-Validation  Sample  N  =  1,578) 


Variable 

Mean 

Standard 

Deviation 

r 

pbis 

r 

bis 

Issue  Non-Issue 

Case  Case 

Issue 

Case 

Non-Issue 

Case 

ASAP 

37.0 

75.3 

511.5 

56.ii 

.  22** 

.36** 

AFQT 

62.1 

66.11 

19.11 

19.11 

.07* 

.11** 

Years 

Ed ucation 

7.2 

7.11 

1 .0 

1 .2 

.06* 

.10«* 

Note : 

Issue  case 

criterion 

is  1  = 

issue  case 

and  2  =  non- 

issue 

case . 

Years  of  education  ranges  from  1  =  1-7  years  to  13  =  doctorate  or 
equivalent . 

*  pC.01 
**  pC.001 


AFQT  scores  and  years  of  education  were  significantly 
correlated  with  the  issue  case  criterion  (biserial  r_’ s  of  .11 
and  .10,  respectively);  however,  these  relationships  were  not 
as  strong  as  that  for  the  ASAP  (biserial  r  =  .36).  Non-issue 
case  personnel  scored  considerably  higher  on  the  ASAP  (mean  = 
75.3)  than  their  issue  case  counterparts  (mean  =  37.0). 

Regression  analyses  were  undertaken  to  examine  whether  the 
combination  of  the  three  predictors  would  improve  prediction. 
In  the  first  regression  analysis,  AFQT  was  forced  into  the 
stepwise  regression  equation  first,  followed  by  years  of 
education  and  ASAP.  In  the  second  regression  analysis,  the 
variables  entered  in  their  natural  order  (ASAP,  AFQT,  then 
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years  of  education)  . 
in  Table  11  . 


The  results  of  these  analyses  are  shown 


TABLE  11 


ASAP  Incremental  Validity 
(Cross-Validation  Sample  N  =  1,578) 


Step 

Variable 

Simple 

r 

Multiple 

R 

Significance 
of  Change 

P< 

(1) 

AFQT 

.11 

.m 

.001 

(2) 

Years 

Education 

.10 

.132 

NS 

(3) 

ASAP 

.36 

.362 

.001 

(1) 

ASAP 

.36 

.356 

.001 

(2) 

AFQT 

.11 

.360 

NS 

(3) 

Years 

Education 

.10 

.362 

NS 

Note 

:  All 

correlations 

have  been 

converted  from  point 

-biserial  to  biserial 

While  the  ASAP  significantly  added  to  the  prediction  when 
AFQT  and  years  of  education  were  entered  first,  neither  AFQT 
nor  years  of  education  significantly  added  to  the  prediction 
when  ASAP  entered  first.  Indeed,  the  prediction  using  ASAP 
alone  (biserial  £  =  .358)  was  almost  identical  to  the 

prediction  obtained  when  using  all  three  variables  in 
combination  (biserial  £  =  .367).  Overall,  the  ASAP  was  a  far 

superior  predictor  to  either  of  the  other  two  measures. 

Ad judi cation  Results 


Table  12  presents  the  results  from  the  adjudication  phase 
for  Army  and  Air  Force  personnel,  broken  down  by  the  issue  case 
status  of  the  background  investigation. 

TABLE  12 

Adjudication  Decision  by  Issue  Case  Status 


Clearance 

Bac  kground 
Issue  Case 

Investigation 
Non-Issue  Case 

Total 

N 

% 

N 

% 

N 

% 

Denied 

7 

(33) 

iji 

(67) 

21 

(1) 

Pending 

Adjudication 

31 

(29) 

77 

(71) 

108 

(5) 

Secret 

17 

( 12) 

■\2H 

(88) 

11|1 

(7) 

Top  Secret/SCI 

W 

(8) 

168H 

(92) 

1831 

(87) 

Note:  Table  includes  only  Army  and  Air  Force  personnel. 
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Only  one  percent  of  the  sample  had  their  clearance  denied 
based  on  information  from  the  background  investigation.  Five 
percent  of  the  sample  had  the  adjudication  decision  still 
pending.  This  category  often  represents  personnel  for  whom  the 
adjudicator  has  requested  additional  information  either  because 
the  overall  investigative  results  are  incomplete  or,  more 
likely,  because  adverse  information  needs  to  be  documented  in 
greater  detail  to  potentially  support  a  clearance  denial.  The 
seven  percent  of  the  sample  in  the  secret  category  could  either 
represent  individuals  whose  new  job  turned  out  not  to  require  a 
top  secret/SCI  clearance,  or,  in  some  cases,  individuals  whose 
background  investigation  yielded  results  that  would  disqualify 
them  from  the  highest  level  clearances  but  would  still  support 
a  secret  clearance.  Eighty-seven  percent  of  the  sample 
received  a  top  secret/SCI  clearance. 

An  examination  of  Table  12  with  regard  to  issue  case 
status  of  the  background  investigation  provides  some 
interesting  findings.  Two-thirds  of  those  personnel  who  had 
their  clearance  denied  did  not  have  issue  case  background 
investigations.  Likewise,  over  70  percent  of  the  personnel  in 
the  pending  adjudication  category  were  not  issue  cases.  These 
results  substantiate  earlier  assertions  that  a  large  number 
(perhaps  as  high  as  2/3)  of  the  personnel  with  adverse 
background  information  are  not  classified  as  issue  cases. 

A  comparison  of  the  ASAP  scores  for  personnel  in  each  of 
the  four  adjudication  categories  is  presented  in  Table  13. 

TABLE  13 

Comparison  of  ASAP  Scores 
by  Different  Adjudication  Categories 


ASAP  Scores8 

Clearance 

N 

Mean 

Denied 

21 

38.8 

Pending 

Adjudication 

108 

47-9 

Secret 

141 

57.3 

Top  Secret/SCI 

1,831 

78.8 

aF  =  20.2,  p  <  .001,  (df  =  3,  2097) 
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Conclusions 


The  objective  of  the  present  study  was  to  conduct  an 
exploratory  study  examining  the  validity  of  behavioral  science 
tests  in  predicting  personnel  security  criteria.  More 
specifically,  an  assessment  was  made  to  determine  the  degree  to 
which  the  ASAP  could  predict  the  issue  case  status  of 
background  investigations.  The  results  were  very  encouraging. 
ASAP  scores  significantly  predicted  the  issue  case  criterion 
and  also  differentiated  between  those  Army  and  Air  Force 
personnel  who  were  granted  a  top  secret/SCI  clearance  versus 
those  personnel  who  received  a  lower  clearance  or  were  denied  a 
clearance.  While  the  results  support  the  potential  use  of 
ASAP-type  instruments  as  prescreens  for  personnel  entering  high 
security  risk  jobs,  a  number  of  caveats  emerge. 

First,  the  present  study  could  not  identify  those 
personnel  who  were  prescreened  out  by  the  service  prior  to  the 
formal  request  for  a  background  investigation.  This  selected- 
out  group  is  especially  important  because  the  greatest 
feasibility  in  using  instruments  such  as  the  ASAP  may  be 
identifying  those  personnel  who  have  a  high  probability  of 
being  prescreened  out  by  the  services.  Administration  of  an 
instrument  such  as  the  ASAP  may  be  far  more  cost  effective  than 
conducting  more  in-depth  prescreening  on  a  total  applicant 
pool.  Future  research  must  determine  whether  or  not 
instruments  such  as  the  ASAP  can  cost-effectively  perform  this 
function . 

A  second  limitation  of  the  current  study  concerned  the  use 
of  issue  cases  as  the  key  criterion.  The  fact  that  two-thirds 
of  the  personnel  who  were  denied  clearances  were  not  issue 
cases  indicates  that  there  is  a  need  to  better  measure  the 
degree  to  which  background  investigations  contain  adverse 
information.  It  is  certainly  too  costly  and  time  consuming  for 
researchers  to  review  and  classify  hard  copy  background  data 
for  the  large  samples  usually  found  in  test  validation  studies. 
One  alternative  may  be  to  have  adjudicators  provide  ratings  of 
the  degree  of  severity  of  the  adverse  background  information  in 
those  cases  where  they  do  not  deny  a  clearance.  These  ratings 
could  be  done  both  for  the  overall  investigation  and  by 
different  content  areas  (e.g.,  involvement  with  law,  substance 
abuse,  financial  problems,  etc.).  Such  ratings  could  then  be 
used  for  research  purposes  only.  Clearly,  the  degree  to  which 
policy  makers  will  accept  the  implementation  of  prescreening 
tests  will  heavily  depend  on  the  reliability  and  validity  of 
both  these  tests  and  the  personnel  security  criteria  used  to 
validate  them. 
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Lastly,  and  perhaps  most  importantly,  the  present  study 
was  conducted  as  an  exploratory  effort.  The  recruiting, 
classification,  and  other  operational  implications  of 
implementing  instruments  such  as  the  ASAP  were  not  examined  in 
depth.  There  is  clearly  a  need  for  a  thorough  and  detailed 
mapping  of  the  prescreening  procedures  and  policies  used  by 
each  of  the  different  services. 

Flyer  (1986)  has  provided  initial  data  for  this  mapping. 
However,  such  a  study  should  expand  on  Flyer's  work  to  include 
detailed  information  for  each  of  the  services  on  different 
organizational  responsibilities,  different  information 
collected,  and  different  decision  processes.  Such  information, 
properly  organized,  could  provide  a  framework  for  identifying 
those  points  in  the  prescreening  process  where  tests  such  as 
the  ASAP  could  provide  the  most  leverage.  It  could  also 
identify  interaction  points  across  the  services  that  might 
provide  opportunities  for  cost  savings  through  improved 
consolidation . 

Despite  the  above  caveats,  the  findings  from  the  current 
study  are  encouraging.  It  appears  that  behavior  science  tests 
such  as  the  ASAP  can  potentially  play  an  important  role  in  the 
prescreening  of  applicants  for  high  security  risk  jobs. 
Implementation  issues  are  critical  but  are  not  insurmountable. 
Future  studies  in  this  area  could  provide  the  operational 
detail  necessary  to  firmly  implement  the  recommendations  of  the 
two  recent  security  panels/commissions  --  that  behavioral 
science  tests,  if  valid,  be  incorporated  into  the  prescreening 
procedures  used  to  approve  security  clearances. 
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